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i 

LISTING OF CLAIMS: 

i 

Claim 1 (Currently Amended): An information input processing computer system for mapping 
gestures to keys of a virtual an invisible virtual keyboard, the system comprising one or several 
cameras, one or more memories with CPU connected to the cameras, and processes running in 
the CPU that associate gesture movements made without touching any touch sensors with typing 
and produce gesture associated textual output, wherein said processes capture gesture images, 
classify e ach typ e of the gesture imag e images into a r e spectiv e on e of a plurality of classes^ 
d e pending on th e typ e of g e stur e , and associate each of the classes with one or more possible 
keys of the invisible keyboard, assign a probability to each of said possibl e k e ys for each of said 
classes, assign a probability to each of the possible keys associated with the class, and integrate 
the probabilities assigned to the possible keys to identify a word for a sequence of gestures. 

Claim 2 (Original): The gesture-key mapping system as in claim 1, where a feedback is 
provided to the user on what kind of keys are associated with the user's gestures. 

Claim 3 (Original): The gesture-key mapping system as in claim 2, where the feedback is 
provided using one or more of the following: displaying keys on a display, playing sounds labels 
for keys, displaying image indicators on a display, playing special sound indicators, projecting 
the keyboard to any surface, and displaying picture of the keyboard with user's hands. 

i 
j 

j Claim 4 (Currently Amended): An information input processing, gesture-key mapping computer 

| system for generating text from hand gestures of a user relative to an invisible virtual keyboard, 

i 
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the system comprising one or several cameras, one or more memories with CPU connected to the 
cameras, and processes running in the CPU that associate gesture movements with typing and 
produce gesture associated textual output, where the gesture-key processing is provided using the 
following modules: 

a) a gesture capturing module that captures gestures, relative to an invisible 
keyboard and made without touching any touch sensors, through camera 
sensors; 

b) a gesture classificator module that classifies each typ e of g e stur e the gestures 
into a resp e ctiv e ono of a plurality of classes depending on the type of g e stur e 
mov e ments ; 

c) an associator module for associating g e stur e each of the classes or soquenc e of 
gesture class e s with one or more possible keys of the invisible keyboard and 
assign s a probability to eaoh of said possibl e k e ys for each of said classes, 
assign a probability to each of the possible keys associated with the class ; and 

d) an integrator module that integrates the probabilities assigned to the possible 
keys to identify a word for a sequence of gestures. 

Claim 5 (Original): The system as in claim 4, where the integrator module includes one or more 
of the following: 



a) language module component that estimate probabilities of word strings 

corresponding to key candidate sequences; 
; b) character frequency module that estimate probabilities of character strings 

corresponding key candidate sequences; 
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c) confusable matrix that estimate how often correct gesture classes are 
confusable with another gesture classes ; 

d) gesture classes probability model that estimate probability of observing a string 
of gesture classes given a sequence of gesture frames; 

e) computation of a probability of production a sequence of keys given a string of 
gesture frames; 

f) generation of a lattice of sequences of keys given sequence of gesture frames; 

g) finding the most probable sequence of keys from the lattice of key candidate 
strings. 

Claim 6 (Original): A system according to Claim 5, wherein each sequence of keys receives a 
probability score, and the sequences of keys that receive low scores are removed and are not 
continuing when new candidates for keys arrive. 

Claim 7 (Previously Presented): A system according to Claim 4, further comprising a gesture 
correlator module to allow to adjust automatically the invisible keyboard to changes in the hand 
positions of the user, and wherein the gesture correlator module acts between the gesture 
classificator module and the associator module to maintain a consistent mapping of gesture 
classes to keys of the invisible keyboard despite said changes in the hand positions of the user. 

Claim 8 (Currently Amended): The method for producing a textual output in which a user makes 
typing like gesture relative to a virtual ; an invisible virtual keyboard made without touching any 
touch sensors and without the presence of a real keyboard and the gestures are associated with 
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the most probable keys that would be typed if a keyboard were presented, said method including 
the steps of using a computer system to map gestures made, without touching any touch sensors, 
to keys of the virtual keyboard, including the steps of running processes on the computer to 
capture gesture images, to classify each typo of the gesture imago images into a resp e ctive on e of 
a plurality of classes^ depending on th e typ e of g e sture, and to associate each of the classes with 
one or more possible keys of the invisible keyboard, assign a probability to oaoh of aoid possibl e 
kevs for each of said classes, assign a probability to each of the possible kevs associated with the 
class, and integrate the probabilities assigned to the possible keys to identify a word for a 
sequence of gestures. 

Claim 9 (Currently Amended): The method for producing a textual output in which a user makes 
typing like gestures relative to a virtual, an invisible virtual keyboard and without the presence of 
a real keyboard and the gestures are associated with the most probable keys that would be typed 

\ if a keyboard were presented, said method including the step of using a computer system to map 

gestures made, without touching any touch sensors, to keys of the virtual keyboard, including the 
step of running processes on the computer to capture gesture images, to classify oaoh typ e of die 

i gesture imago images into a respective on e of a plurality of classes A depending - on the typ e of 

gesture, and to associate each of the classes with one or more possible keys of the invisible 
keyboard, assign a probability to each of said possible koys for each of said classes, assign a 
probability to each of the possible kevs associated with the class , and integrate the probabilities 
assigned to the possible keys' to identify a word for a sequence of gestures, and wherein the 
probability is computed using HMM. 

■: 

! 
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Claim 10 (Currently Amended): A method of typing using a virtual keyboard having a multitude 
of virtual keys, comprising the steps: 

making typing gestures relative to a virtual, an invisible virtual keyboard made 
without touching any touch sensors and without any real keyboard; 

sensing the typing gestures; and 

producing, from the sensed typing gestures, gesture associated textual output 
including the step of running processes on a computer to capture gesture images, classify eaeh 
type of the gesture imag e images into a r e spective one of a plurality of classes* depending on th e 
typ e- of g e stur e , and to associate each of the classes with one or more possible keys of the 
invisible keyboard, assign a probability to e ach of said possible k e ys for each of said classes, 
assign a probability to each of the possible keys associated with the class , and integrate the 
probabilities assigned to the possible keys to identify a word for a sequence of gestures. 

Claim 1 1 (Original): A method according to Claim 10, wherein the typing gestures are made by 
a person, and further comprising the steps of providing feedback to the person on texture output 
associated with the gestures. 

Claim 12 (Original): A method according to Claim 11, wherein the step of providing feedback 
includes the step of displaying an image of typing keys associated with the gestures. 

Claim 13 (Cancelled). 
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Claim 14 (Currently Amended): A method of typing using a virtual an invisible virtual 
keyboard, comprising the steps: 

making typing gestures relative to a virtual, an invisible virtual keyboard made 
without touching any touch sensors and without any real keyboard; 

sensing the typing gestures; and 

producing, from the sensed typing gestures, gesture associated textual output: and 
wherein the producing step includes the steps of classifying oaoh type of gestur e the gestures into 
a r e spective on e of a plurality of classes^ depending on th e typo of gesture, associating each of 
said classes with one or more possible keys of the invisible keyboard, as s igning a probability to 
oooh of said possibl e k e ys for each of said classes, assign a probability to each of the possible 
keys associated with the class, and integrating the probabilities assigned to the possible keys to 
identifying a word for a sequence of gestures. 

Claim 15 (Original): A method according to Claim 14, wherein the producing step further 
includes the step of associating gesture classes with individual typing keys. 

Claim 16 (Previously Presented): A method according to Claim 14, further comprising 
providing training data in words or sentences with certain timing data. 

Claim 17 (Currently Amended): A typing system using a virtual, an invisible keyboard, 
comprising 

means for sensing typing gestures made relative to a virtual; an invisible virtual 
keyboard made without touching any touch sensors and without any real keyboard; and 
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means for producing, from the sensed typing gestures, gesture associated textual 
output said producing means including a computer and processes running on the computer to 
capture gesture images, to classify e ach type of gesture imag e the gesture images into a 
respectiv e ono of a plurality of classes* d e p e nding on th e typo of gestur e , and associate each of 
the classes with one or more possible keys of the invisible keyboard, assign a probability to e ach 
of said possible keys for each of said classes, assign a probability to each of the p ossible keys 
associated with the class, and integrate the probabilities assigned to the possible keys to identify 
a word for a sequence of gestures. 

Claim 1 8 (Original): A system according to Claim 17, wherein the typing gestures are made by a 
person, and further comprising means for providing feedback to the person on texture output 
associated with the gestures. 

Claim 19 (Original): A system according to Claim 18, wherein the means for providing 
feedback includes means for displaying an image of typing keys associated with the gestures. 

Claim 20 (Cancelled). 

Claim 21 (Currently Amended): A typing system using a virtual an invisible virtual keyboard, 
comprising 

means for sensing typing gestures made relative to a virtual, invisible keyboard 
made without touching any touch sensors and without any real keyboard; and 

g G:\IbmU 05U33 1 7\amend\l 33 J 7.am7.doc 

PACE 13/18 * RCVD AT 9/14/2005 4:14:07 PM [Eastern Daylight Time] " SVR:USPTO-EFXRF-6/35 " DNl3:2738300 " C8ID:5167424366 * DURATION <mm-ss):05-24 



9-14-05; 4: 13PM;SSMP FAX 



; 5 167424366 



# 14/ 18 



means for producing, from the sensed typing gestures, gesture associated textual 

output; and 

wherein the producing means includes means for classifying each typ e of gestur e 
the gestures into a r e sp e ctiv e on e of a plurality - ef classes^ depending on the typ e of gestur e , 
associating each of said classes with one or more possible keys of the invisible keyboard, 
assigning a probability to e ach of said possible kovs for each of said classes, assign a probability 
to each of the possible keys associated with the class , and integrating the probabilities assigned 
to the possible keys to identifying a word for a response of gestures. 

Claim 22 (Original): A system according to Claim 21, wherein the producing means farther 
includes means for associating gesture classes with individual typing keys. 

Claim 23 (Previously Presented): A method according to Claim 10, wherein the step of running 
processes on the computer includes the step of mapping the sensed typing gestures to keys of the 
keyboard based on a statistical machine that interprets sequences of typing like gesture classes as 
intended words based on user typing gesture models. 

Claim 24 (Previously Presented): A method according to Claim 10, wherein gesture input is 
represented as wave forms that are digitized and clustered in gesture frames, and then processed 
by HMM machines that represent difference models for different typing patterns; and the step of 
running processes includes the step of using two statistical components, a first statistical 
component to map gestures to keys, and a second, language model component to map keys to 
letters and words. 
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